Papers with single-step tasks
PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. |
| Approach: | They propose a method to optimize prompts for LLM-driven multi-step tasks using a human-designed feedback rule. |
| Outcome: | The proposed method outperforms human-engineered prompts and several other prompt optimization methods on 11 representative multi-step tasks. |
SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | SeqPO-SiMT is a new policy optimization framework for simultaneous machine translation that combines a tailored reward with a single step task. |
| Approach: | They propose a new policy optimization framework that defines the simultaneous machine translation task as a sequential decision making problem with a tailored reward. |
| Outcome: | The proposed framework outperforms the supervised fine-tuning model by 1.13 points while reducing the Average Lagging by 6.17 in the NEWSTEST2021 En Zh dataset. |
TurnBench-MS: A Benchmark for Evaluating Multi-Turn, Multi-Step Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks focus on single-turn or single-step tasks, failing to capture iterative reasoning in real-world settings. |
| Approach: | They propose a benchmark that evaluates multi-turn, multi-step reasoning through an interactive code-breaking task inspired by the "Turing Machine Board Game" the best model achieves 84% accuracy in Classic mode, but performance drops to 18% in Nightmare mode. |
| Outcome: | The new benchmark evaluates multi-turn, multi-step reasoning through an interactive code-breaking task inspired by the "Turing Machine Board Game" the best model achieves 84% accuracy in Classic mode, but performance drops to 18% in Nightmare mode. |